Skip to main content

Managing Policies

This section outlines how to manage, apply, edit, and retrain custom content policies.

Policy Page​

Navigate to the Policies tab to access the policy registry - a list of all policies you have created for guardrailing. Through this page, you can view policy information and apply them to models.

Policy Instance​

Click on the Manage Policy button to view the policy instance page for a specific policy. This page provides comprehensive informaiton about the policy definition, training data, and incoming feedback. This page can be used to edit and manage individual policies.

Policy Registry

Training Content Policies​

Before applying a content policy, the policy must first be fine-tuned on the synthetically generated training dataset. To train a policy, click the train policy button within the policy management page. The policy will then move to a state of 'Training Queued' and then 'Training Policy'.

Train Policy

Policy Versioning​

When training your policy, you will be asked to create a new policy version. Policy Versioning allows you to track changes made to policies and revert to previous versions if necessary.

Version Policy

Version Policy

Editing Policies​

You can edit a policy’s definition, training data, and feedback at any time. Here’s how to manage those edits:

Editing Policy Definitions​

Policy definitions can be edited from the Policy Management page. Here, edits can be made to the title, description, or associated behaviors. Allowed and disallowed behaviors can also be added and removed. After editing the policy, regenerate the training data to ensure that it aligns with the updated definition.

Policy Editing

Policy Editing

Adding Training Data​

Training data can also directly be added to a policy by clicking the 'Add Data' button in the Training Data tab. Here, data can either be provided one at a time by providing the example and compliance status or by uploading a file containing a larger set of datapoints.

Adding Data

Editing Datapoints​

Existing training data can directly be edited by clicking into the particular datapoint. Here, the datapoint text or compliance status can be edited.

Adding Data

Adding Feedback Data​

After deploying a policy, you can collect feedback on how it performs in the live environment:

  • Navigate to the Monitoring Log to view feedback on datapoints
  • This feedback can be applied to the policy for continuous improvement

Leaving Feedback

Applying Feedback

Retraining Policies​

Whenever you make edits to a policy (such as modifying its definition or adding new training data), you must retrain the policy to ensure that the changes are incorporated. To retrain, navigate to the training rab and click train policy.